Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V

In this paper, we critically evaluate the capabilities of the state-of-the-art multimodal large language model, i.e., GPT-4 with Vision (GPT-4V), on Visual Question Answering (VQA) task. Our experiments thoroughly assess GPT-4V's proficiency in answering questions paired with images using both pathology and radiology datasets from 11 modalities (e.g. Microscopy, Dermoscopy, X-ray, CT, etc.) and fifteen objects of interests (brain, liver, lung, etc.). Our datasets encompass a comprehensive range of medical inquiries, including sixteen distinct question types. Throughout our evaluations, we devised textual prompts for GPT-4V, directing it to synergize visual and textual information. The experiments with accuracy score conclude that the current version of GPT-4V is not recommended for real-world diagnostics due to its unreliable and suboptimal accuracy in responding to diagnostic medical questions. In addition, we delineate seven unique facets of GPT-4V's behavior in medical VQA, highlighting its constraints within this complex arena. The complete details of our evaluation cases are accessible at https://github.com/ZhilingYan/GPT4V-Medical-Report

Enhancing Building Semantic Segmentation Accuracy with Super Resolution and Deep Learning: Investigating the Impact of Spatial Resolution on Various Datasets

The development of remote sensing and deep learning techniques has enabled building semantic segmentation with high accuracy and efficiency. Despite their success in different tasks, the discussions on the impact of spatial resolution on deep learning based building semantic segmentation are quite inadequate, which makes choosing a higher cost-effective data source a big challenge. To address the issue mentioned above, in this study, we create remote sensing images among three study areas into multiple spatial resolutions by super-resolution and down-sampling. After that, two representative deep learning architectures: UNet and FPN, are selected for model training and testing. The experimental results obtained from three cities with two deep learning models indicate that the spatial resolution greatly influences building segmentation results, and with a better cost-effectiveness around 0.3m, which we believe will be an important insight for data selection and preparation

Evaluation of oral Lanzhou lamb rotavirus vaccine via passive transfusion with CD4+/CD8+ T lymphocytes

AbstractLanzhou Lamb derived Rotavirus (RV) Vaccine (namely LLR) for children is only used in China. Since there were no reports on evaluation of LLR, even the data of phase IV clinical trial, we proceed the evaluation of LLR through focusing on T-cell to investigate whether LLR could induce the potential function involving in protection as a vaccine. Four groups of nude mice were transfused with CD4+/CD8+ T-cells isolated from LLR-immunized (primed) and LLR-unimmunized (naïve) mice via intraperitonea (i.p.) respectively. Consequently, the adoption mice were challenged with mice-origin wild rotavirus EDIM (Epizootic Diarrhea of Infant Mice) by intragastric administration. Series of fecal/serum samples were collected and viral shedding, then serum IgA/IgG and secreted IgA were assayed. Compared to the mice transfused with T lymphocytes from naïve mice, the nude mice transfused with CD4+ T lymphocytes from primed mice induce fecal and serum IgA increasing more rapidly, and have a shorter duration of virus shedding too. Whereas, no significant difference in virus clearance was found between the mice transfused with CD8+ T lymphocytes isolated from primed and naïve mice. Therefore, we cleared the distinct roles of transfused CD4+/CD8+ T lymphocytes for rotavirus clearance in nude mice, that the viral clearance conducted by CD4+ T lymphocytes. Meanwhile, it has ability to help induction of LLR specific immunogenicity. Comparing with the transfusion of cell from primed and naïve mice, LLR can induce CD4+ T lymphocytes memory which is a potential index to reflect the immunogenicity and protection, while CD8+ T lymphocytes remove rotavirus by CTL with little memory ability

CellMix: A General Instance Relationship based Method for Data Augmentation Towards Pathology Image Classification

In pathology image analysis, obtaining and maintaining high-quality annotated samples is an extremely labor-intensive task. To overcome this challenge, mixing-based methods have emerged as effective alternatives to traditional preprocessing data augmentation techniques. Nonetheless, these methods fail to fully consider the unique features of pathology images, such as local specificity, global distribution, and inner/outer-sample instance relationships. To better comprehend these characteristics and create valuable pseudo samples, we propose the CellMix framework, which employs a novel distribution-oriented in-place shuffle approach. By dividing images into patches based on the granularity of pathology instances and shuffling them within the same batch, the absolute relationships between instances can be effectively preserved when generating new samples. Moreover, we develop a curriculum learning-inspired, loss-driven strategy to handle perturbations and distribution-related noise during training, enabling the model to adaptively fit the augmented data. Our experiments in pathology image classification tasks demonstrate state-of-the-art (SOTA) performance on 7 distinct datasets. This innovative instance relationship-centered method has the potential to inform general data augmentation approaches for pathology image classification. The associated codes are available at https://github.com/sagizty/CellMix

Preparation of fluorescence-encoded microspheres in a core-shell structure for suspension arrays

Fluorescence-encoded microspheres are widely used in the detection and analysis of biological molecules, especially in suspension arrays. Here, we report an efficient strategy for the preparation of fluorescence-encoded polystyrene microspheres with desirable optical and surface properties. The micron-sized, monodisperse polystyrene seed beads were first synthesized by dispersion polymerization. Then, dye molecules and carboxyl functional groups were copolymerized on the surface of the seed beads by forming a core-shell structure. Rhodamine 6G (R6G) was used as a model dye molecule to prepare the fluorescent beads, and the fluorescence intensity of the beads can be precisely controlled by adjusting the quantity of R6G. These fluorescent beads were characterized by environmental scanning electron microscopy, laser scanning confocal microscopy, and spectrofluorometry. The differences of the fluorescence spectra between fluorescent beads and R6G in solution were investigated. Twelve kinds of fluorescent beads encoded with different R6G fluorescence intensities were prepared, and they can be clearly distinguished on a conventional flow cytometer. Furthermore, the encoded beads are stable in water and resistant to photobleaching, which is crucial for their potential applications in diagnostic assays and imaging. Detection of human alpha fetoprotein antigen via a sandwich microsphere-based immunoassay yielded a detection limit of 80 pg mL(-1), demonstrating that the fluorescence-encoded microspheres synthesized herein are efficient in serving as the microcarriers in suspension arrays. As both the encoding and functionalizing procedures are made simultaneously, the newly designed technique is extremely simple and time-saving. Moreover, it could be readily applicable to the preparation of a wide size range of fluorescent particles made by polymerization.National Natural Science Foundation of China [20675070]; Program for New Century Excellent Talents in University [NCET-07-0729]; NFFTBS [J0630429]; Scientific Research Foundation ; State Education Ministr

Research on the derated power data identification method of a wind turbine based on a multi-Gaussian-discrete joint probability model

This paper focuses on how to identify normal, derated power and abnormal data in operation data, which is key to intelligent operation and maintenance applications such as wind turbine condition diagnosis and performance evaluation. Existing identification methods can distinguish normal data from the original data, but usually remove power curtailment data as outliers. A multi-Gaussian-discrete probability distribution model was used to characterize the joint probability distribution of wind speed and power from wind turbine SCADA data, taking the derated power of the wind turbine as a hidden random variable. The maximum expectation algorithm (EM), an iterative algorithm derived from model parameters estimation, was applied to achieve the maximum likelihood estimation of the proposed probability model. According to the posterior probability of the wind-power scatter points, the normal, derated power and abnormal data in the wind turbine SCADA data were identified. The validity of the proposed method was verified by three wind turbine operational data sets with different distribution characteristics. The results are that the proposed method has a degree of universality with regard to derated power operational data with different distribution characteristics, and in particular, it is able to identify the operating data with clustered distribution effectively

MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation

The Segment Anything Model (SAM), a foundation model for general image segmentation, has demonstrated impressive zero-shot performance across numerous natural image segmentation tasks. However, SAM's performance significantly declines when applied to medical images, primarily due to the substantial disparity between natural and medical image domains. To effectively adapt SAM to medical images, it is important to incorporate critical third-dimensional information, i.e., volumetric or temporal knowledge, during fine-tuning. Simultaneously, we aim to harness SAM's pre-trained weights within its original 2D backbone to the fullest extent. In this paper, we introduce a modality-agnostic SAM adaptation framework, named as MA-SAM, that is applicable to various volumetric and video medical data. Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments while preserving the majority of SAM's pre-trained weights. By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data. The effectiveness of our method has been comprehensively evaluated on four medical image segmentation tasks, by using 10 public datasets across CT, MRI, and surgical video data. Remarkably, without using any prompt, our method consistently outperforms various state-of-the-art 3D approaches, surpassing nnU-Net by 0.9%, 2.6%, and 9.9% in Dice for CT multi-organ segmentation, MRI prostate segmentation, and surgical scene segmentation respectively. Our model also demonstrates strong generalization, and excels in challenging tumor segmentation when prompts are used. Our code is available at: https://github.com/cchen-cc/MA-SAM